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fullness at t, r (b/). plus the number of bits entering the 
decoder VBV buffer between t, r and 



minus the number of bits removed form the decoder buffer 
between I* and t,. The number of bits removed is the sum of 
encoder buffer occupancy at t* immediately before adding 10 
VOP i (eb, r ) and decoder buffer occupancy at t * (b'J 
because all bitstream data prior to VOP i must be consumed 
before VOP i can be decoded. The later two quantities 
represent bitstream data prior to VOP i since VOP i has not 
been added to the encoder buffer. Therefore, total bits in the J5 
decoder buffer are bounded by 



Which yields 



r 



25 



Therefore, encoder_buffer_Jullness at t,' is lower 
bounded by 



The same arguments given above about the constant delay 
channel can be applied here. Also, for the type of channels 35 
which have known maximum transmission rate R^/,^ T, 
can be set to be a upper bound of 



r 



40 



The bounds T, and T 2 are checked in the rate-control 
algorithm and the corrective action are to perform bit- 
allocation of the VOPs and adjust quantization levels of the 45 
coding units (e.g. VOP, macroblocks). 

The encoder must take the following corrective action if 
the simulated decoder VBV buffer gets too full or too empty: 

1. If the simulated decoder VBV buffer becomes too full 
(i.e. the encoder VRV huff-*- i: *-z iv^lj), uic cucouer 50 
can correct the problem by: 

(a) reducing the quantization level to generate large 
VOPs, or 

(b) outputting stuffing bits at the end of the VOP. Note 
that generating larger VOPs reduces the decoder 55 
VBV occupancy. 

2. If the simulated decoder VBV becomes too empty (i.e. 
the encoder VBV buffer is too full), the encoder can 
correct the problem by: 

(a) increasing quantization levels to generate fewer 60 
bits, or 

(b) delaying the generation of the next VOP (often 
called skipping VOP), or 

(c) zeroing the high frequency coefficients to reduce the 
number of bits/VOP generated. 65 

It should now be appreciated that the present invention 
provides a video rate buffer model for bounding the memory 
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requirements of a video decoder in a push dataflow scenario. 
The rate buffer model of the present invention constrains the 
video encoder to producing bitsrreams that are decodable 
with a predetermined buffer memory size. Thus, push data- 
flow applications are efficiently accommodated. 
What is claimed is: 

1. An encoder apparatus for enabling a push dataflow 
bitstream without causing a modeled data buffer of prede- 
termined mem or>' size for the pushed data at a decoder to 
overflow or underflow, comprising: 

a processor adapted to encode data to provide the bit- 
stream for communication to a decoder; wherein: 
the encoded data comprises at least one video or visual 
object (VO) with at least one video or visual object 
layer (VOL), including an associated header, fol- 
lowed by at least one video or visual object plane 
(VOP); 

a field in the VOL header designates an occupancy of 
the buffer just before removal of the first VOP 
following the VOL header from the buffer; and 

said processor uses a simulation to simulate the decoder 
buffer and controls the bitstream in response to the 
simulation to preclude overflow or underflow of the 
decoder buffer. 

2. The apparatus of claim 1, wherein: 

when the buffer is initially empty, the occupancy field is 
examined to determine an initial occupancy of the 
buffer before decoding the initial VOP. 

3. The apparatus of claim 1, wherein: 

the processor provides a flag to control the inclusion of at 
least one field in the VOL header when equivalent 
information is not present in an encapsulating system 
multiplex. 

4. The apparatus of claim 3, wherein: 

the flag allows a visual elementary stream of the bitstream 
as a standalone entity to specify a buffer model. 

5. The apparatus of claim 3, wherein: 

the at least one field whose inclusion in the VOL header 
is controlled by the flag designates a VOP rate of the 
bitstream. 

6. The apparatus of claim 3, wherein: 

the at least one field whose inclusion in the VOL header 
is controlled by the flag designates a peak bit rate of the 
bitstream. 

7. The apparatus of claim 3, wherein: 

the at least one field whose inclusion in die VOL header 
is controlled bv the flna ^ci<rn^ : •■■hi-.i. VOL 
contains at least one B-VOP. 

8. The apparatus of claim 3, wherein: 

the at least one field whose inclusion in the VOL header 
is controlled by the flag designates the size of the 
modeled buffer. 

9. The apparatus of claim 3, wherein: 

the at least one field whose inclusion in the VOL header 
is controlled by the flag designates said field that 
designates the occupancy of the buffer. 

10. The apparatus of claim 1, wherein: 

the video or visual object comprises a video object. 

11. The apparatus of claim 1, wherein: 

the video or visual object comprises a still texture object. 

12. The apparatus of claim 1, wherein: 

the video or visual object comprises a mesh object. 

13. The apparatus of claim 1, wherein: 

the video or visual object comprises a face object. 
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14. The apparatus of claim 1. wherein: 

' the encoded data comprises a plurality of VOLs, and a 
decoder buffer model is applied independently to each 
VOL using buffer size and rate functions particular to 
each VOL. * 5 

15. The apparatus of claim 1. wherein: 

the bitstream is compatible with an MPEG4 video coding 
standard. 

16. The apparatus of claim 1, wherein: 

additional fields are provided in respective subsequent io 
VOL headers to designate respective subsequent buffer 
occupancy levels. 

17. The apparatus of claim 16, wherein: 

the processor maintains a difference between the addi- 
tional fields in the subsequent VOL headers and a 15 
running cumulative buffer occupancy just before 
removal of a VOP from the buffer within a tolerance. 

18. The apparatus of claim 1, wherein: 

the processor determines a size (d.) of a current VOP as 
a number of bits extending to the last bit of the current 
VOP and starting from either the last bit of the previous 
VO or the first bit of a start code for the first VOP of 
the encoded data. 

19. The apparatus of claim 1, wherein: 

the processor determines a decoding time t, of an ith VOP, 
wherein: 25 

(a) t~r ( - if the VOL contains no B-VOPs, where t, is a 
composition time of the ith VOP, and 

(b) t~c r m„ when the ith VOP is an anchor VOP, and 
m f accounts for a delay of at least one immediately 
subsequent B-VOP to be composited or presented. 30 

20. The apparatus of claim 1, wherein: 

the processor determines a decoding time x i of an ith VOP, 
wherein: 

(a) t=r, if the VOL contains no B-VOPs, where t. is a 
presentation time of the ith VOP when the decoder is 35 
a no-compositor decoder, and 

(b) t~r r m,., when the ith VOP is an anchor VOP, and 
m, accounts for a delay of at least one immediately 
subsequent B-VOP to be composited or presented. 

21. The apparatus of claim 1, wherein: 
said pushed data flow comprises video data that includes 

intraframe (1), prediction (P) and bi-directional (B) 
video object planes (VOPs). 

22. The apparatus of claim 21, wherein said processor 
controls said bitstream by at least one of: 

allocating bits between different VOPs, and 
adjusting quantization levels of coding units forming said 
VOPs. 

23. The aDDarams r»f Moi™ 

the modeled data buffer comprises a visual or video 
buffering verifier (VBV) buffer. 

24. The apparatus of claim 23, wherein: 

said processor monitors the VBV buffer and, when the 
simulation indicates that the VBV buffer has or will 55 
become too full, a quantization level for said coding 
units is reduced. 

25. The apparatus of claim 23, wherein: 

said processor monitors the VBV buffer and, when the 
simulation indicates that the VBV buffer has or will 60 
become too empty, said quantization level for said 
coding units is increased. 

26. The apparatus of claim 23, wherein: 

said processor monitors the VBV buffer and, when the 
simulation indicates that the VBV buffer has or will 65 
become too empty, the generation of the next VOP is 
delayed. 
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27. The apparatus of claim 23, wherein: 

said processor monitors the VBV buffer and, when the 
simulation indicates that the VBV buffer has or will 
become too empty, high frequency coefficients of said 
coding units are zeroed to reduce the number of bits 
generated per VOP. 

28. The apparatus of claim 23. wherein: 

said processor monitors the VBV buffer and, when it is 
determined that the VBV buffer has or will become too 
full, stuffing bits are added to the end of at least one 
VOP. 

29. The apparatus of claim 1, wherein: 

the processor includes an encoder buffer for receiving the 
encoded data prior to providing the bitstream there- 
from: and 

the processor controls the rate of the bitstream such that 
the fullness of the encoder buffer eh/" after encoding an 
ith VOP is upper bounded by 



where t/ is the time of starting to encode the ith VOP, 
L is the time difference between the encoding time t * 
and the decoding time of the ith VOP. d,. is the amount 
of coded data for the ith VOP, and K voLdecoder {\) is the 
instantaneous channel bit rate seen by the decoder. 

30. The apparatus of claim 1, wherein: 

the processor includes an encoder buffer for receiving the 
encoded data prior to providing the bitstream there- 
from; and 

the processor controls the rate of the bitstream such that 
the fullness of the encoder buffer eh/" after encoding an 
ith VOP is upper bounded by 



where t * is the time of starting to encode the ith VOP, 
L is the time difference between the encoding time if 
and the decoding time of the ith VOP, d, is the amount 
of coded data for the ith VOP, and R v0/ . wc<7 ^ r (t) is the 
instantaneous channel bit rate seen by the encoder. 

31. The apparatus of claim 1, wherein: 

the processor includes an enrrvW hnffiw f ?r r:;c; , ; 116 c ; lt 
encoded data prior to providing the bitstream there- 
from; and 

the processor controls the rate of the hjrerream such that 
the fullness of the encoder buffer eb/ after encoding an 
ith VOP is lower bounded by 



where t * is the time of starting to encode the ith VOP, 
L is the time difference between the encoding time t/ 
and the decoding time of the ith VOP, B is the size of 
the decoder buffer, and KoUecodM is toe instanta- 
neous channel bit rate seen by the decoder. 

32. The apparatus of claim 1. wherein: 

the processor includes an encoder buffer for receiving the 
encoded data prior to providing the bitstream there- 
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from; and the processor controls the rate of the bit- 
stream such that the fullness of the encoder buffer eh ' 
aner encod.ng an ith VOP is lower bounded by 
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Of. 



where i, is the time of staning to encode the ith VOP 

' S r' 1 ™ dlfferen « between the encoding time t ; 10 
and the decoding time of the ith VOP. B is the size of 

eous'ch f Md is ^ 

neous channel bit rate seen by the encoder 

33. An encoding method for enabling a push dataflow 
bttstream wtthout causing a modeled dam buffer of prede " 
termmed memory size for the pushed data at a decoder 0 
overflow or underflow, comprising the steps of- 

encodmg daut to provide the bitstream for communication 
to a decoder; wherein: 

the encoded data comprises at least one video or visual ^ 
aver (VOL), mcluding an associated header fol- 
(VOP): a^d ^ ^ 0r ^ ° bjeCt p,ane 

a field in the VOL header designates an occupancy of * 
* buffer just before removal of the first VOP 
followmg the VOL header from the buffer- and 

US 'c n onl Si r K ti0n 10 SimU ' a,e ^ decoder buffer and 
SSL T t,m in reSp ° nse 10 the simul =«ion to 30 
preclude overflow or underflow of the decoder buffer. 

34. The method of claim 33, wherein: 

when the buffer is initially empty, the occupancy field is 

lT n ? f t0 , detennine M initial ««« of th 

buffer before decoding the initial VOP 35 
^ 3>. The method of claim 33, comprising the further step 

providing , flag ,o control the inclusion of at least one 
field n the VOL header when equivalent mfotmation 
no. present in an encapsulating system multiplex. 40 

36. The method of claim 35, wherein: 
the flag allows a visual elementary stream of the bitstream 

as a standalone entity to specify a buffer model 

37. The method of claim 35, wherein: 
the at least one field whose inclusion in the VOL header ' 5 

« controlled by the flag designates a VOP rate of he 
oiistream. 

38. The method of claim 35, wherein: 
-~ «; L«* uw i,eia wnose inclusion in the VOL header 50 

SSST by flag designates 3 peak bit n * 

— «..u, uu wj uioiiii j?>, wnerein: 
the at least one field whose inclusion in the VOL header 
is controlled by the flag designates whether the VOL 55 
contains at least one B-VOP. 

40. The method of claim 35, wherein: 
the at least one field whose inclusion in the VOL header 

'JSXLZ* St. 

41. The method of claim 35, wherein: 
the at least one field whose inclusion in the VOL header 

» controlled by the flag designates said field th 
designates the occupancy of the buffer 

42. The method of claim 33. wherein: 65 
the video or visual object comprises a video object. 



43. The method of claim 33. wherein: 
the vkleo or visual object comprises a still texture object 

44. The method of claim 33, wherein: 
the video or visual object comprises a mesh object 
4s. The method of claim 33. wherein: 
the video or visual object comprises a face object 

46. The method of claim 33. wherein: 
the encoded data comprises a plurality of VOLs, and a 

VOL uT U ? r r del iS 3PPlied inde P endenII - v <° «ch 
each VOL "* fUnCti ° nS Pa " icular 10 

47. The method of claim 33, wherein: 

* e mS5F " COmpatible with m MPEG -4 v '1co coding 

48. The method of claim 33. wherein: 

add Vo, n hl fi H e ' dS , "f Pr ° Vided in rCSpective ^sequent 
VOL headers to designate respective subsequent buffer 
occupancy levels. 
J9. The method of claim 48, comprising the further step 



of: 



maintmmng a difference between the additional fields in 
die subsequent VOL headers and a running cumulative 
buffer occupancy just before removal of a VOP from 
the buffer within a tolerance 
^ 50. The method of claim 33, comprising the further step 

determining a size (d f ) of a cmrent VOP as a number of 
bits extending to the last bit of the cunent VOP and 
starting from either die last bit of the previous VO or 
the first bit of a start code for the first VOP of the 
. encoded data. 

51. The method of claim 33, comprising the further step 

determining a decoding time t ( of an ith VOP, wherein- 

(a) 1=1, if the VOL contains no B-VOPs, where t is a 
composition time of the ith VOP, and 

(b) t,=r-m„ when the ith VOP is an anchor VOP and 
m accounts for a delay of at least one immediately 
subsequent B-VOP to be composited or presented 

52. The method of claim 33, comprising the former step 



determining a decoding time t, of an ith VOP, wherein- 

(a) t=r,. if the VOL contains no B-VOPs, where t- is a 
presentation time of the ith VOP when the decoder is 
a no-compositor decoder, and 

(b) t=T ( .-m„ when the ith VOP is an anchor VOP and 
m, accounts for a delav of at le»« » 
suDsequent B-VOP to be composited or present 

53. The method of claim 33, wherein: 

said pushed d.ifn Ami; : j. , . 

i _ ui vom F rijca video aata mat inchwW 

uDinoe (1) prediction (P) and bi-directional" "(B) 
video object planes (VOPs) 

54. The method of claim 53, comprising the further steo 
of controlling said bitstream by at least one of- P 

allocaung bits between different VOPs and 

adjustmg quantization levels of coding units forming said 

55. The method of claim 54, wherein- 
the modeled data buffer comprises a visual or video 

buffering verifier (VBV) buffer 
J6. The method of claim 55, comprising the further step 

monitoring the VBV buffer and, when the simulation 
m icates that the VBV buffer has or will become! 
full, reducing a quantization level for said coding units. 
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^ 57. The method of claim 55 comprising ,he further step 

monitoring the VBV buffer and. when the simulation 
■ndtcates that the VBV buffer has or will become to^ 
em^mcreasing . « Zation level for s 

oft* m meth ° d ° f daim 5S - ^P-ing the funher step 

monitoring the VBV buffer and, when the simulation 
■"dotes ** "« VBV buffer has or will be ome 00 
empty, delaying the generation of the next VOR 

oft' 9 ' m meth ° d ° f daim «■ uprising the further step 

15 

monitoring the VBV hnffrr ™^ u 

^^^^^ 
empty, zeroing high frequency coefficients of Z 
-ding umts to reduce the number of bits generated £ 30 

oft 60 ' ^ ^ ° f claim 5S - comprising the father step 
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"T^l*]™ °i the bi[ ™ -h that the fullness 



r 



R*ol. accede At)dl- 5, 



where t,' is the time of staning to encode the ith VOP 

J t r 6 benV6en encodtn, I?'; 
and the deC0 d ln g time of the ith » [ ™ £ 

the decoder buffer, and R ic 1 °' 

neoiic rhmn^i k-, *>-»'.*<w<-rO) is the instanta- 
neous channel b,t rate seen by the decoder. 

""Pnonsfttfmhetsiep of. 



herefrom, comprising the further step of: 30 
coaling the rate of the bitstream such that the fullness 



&"i»t.enroderU)dt-B. 



RvolMceodeMdt-d;, 



35 



45 



where tr is the time of starting to encode the ith VOP 
L» fc one difference between the encoding dme r 
and the decoding time of the ith VOP d i. Z 

"of ^ °J me bitStream such mat *e fullness 50 
of the encoder buffer eb,< after encoding an ith VOP s 
upper bounded by s 



40 



Ri-ol,entod e rti)dt-d h 



55 



l t h ■ , ir e of stanii,g to enc ° de ** «* vop 

' S f\ tlme dlff erence between the encodin- time , ' 60 
and the decoding time of the ith VOR d is me° amount 



where t f ,s the time of starting to encode the ith VOP 
L ■* the tune difference between the encoding time 
and the decoding time of the ith VOR B is the Tze of 
the decoder buffer, and R , m ic ,L ° f 
neonc rh->n n »i k-. ^w/'mcMrMt) is the instanta- 
neous channel b.t rate seen by the encoder. . 
65. A decoder apparatus, comprising 
a data buffer of predetermined memory size; and 
means for receiving a push dataflow bitstream that is 

^bSvo'r r P f S6S 31 km °" e video or visual 

25 (VOL) ZS™ °" e Vide ° ° r ViSUal ol * c < 
wyer ( VUL), includmg an associated header fnl 

3 1" " w*f , V ? L head6r desi ^ ates an occupancy of 
*e modeled buffer jus, before removal ofTels 

buffer ^ fc V ° L header from *e modeled 

66. The apparatus of claim 65, wherein- 

d6d d f 13 COm P™* a nag that control the inc.,,. 

ine aDnaramc ftfou;^ cc ...i 

. , wu, wuerein: 

as fla /l 0 dT WSUaI elementai > stream of the bitstream 
as a s^ndalone enuty to specify a buffer model 

68. The apparatus of claim 66, wherein' 

^T,?", 6 ? e ' d Wh0se inclusi °n in the VOL header 
,« by the flag designates a VOP ££E 

69. The apparatus of claim 66, wherein- 

the at least one field whose inclusion in the VOL header 

70. The apparatus of claim 66, wherein- 

me at least one field whose inclusion in the VOL 

cZT»1 by Ae flag fc ^S? 0 

contains at least one B-VOP. 
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71. The apparatus of claim 66, wherein: 

the at least one field whose inclusion in the VOL header 
is controlled by the flag desig nates me size of die 
modeled buffer. 

72. The apparatus of claim 66. wherein: 

the at least one field whose inclusion in the VOL header 
is controlled by the flag designates said field that 
designates the occupancy of the buffer. 

73. The apparatus of claim 65, wherein: 

the video or visual object comprises a video object. 

74. The apparatus of claim 65, wherein: 

the video or visual object comprises a still texture object. 

75. The apparatus of claim 65, wherein: 

the video or visual object comprises a mesh object. 

76. The apparatus of claim 65. wherein: 

the video or visual object comprises a face object. 

77. The apparatus of claim 65, wherein: 

the encoded data comprises a plurality of VOLs, and a 
decoder buffer model is applied independently to each 
VOL using buffer size and rate functions particular to 
each VOL. 

78. The apparatus of claim 65, wherein: 

the bitstream is compatible with an MPEG4 video coding 
standard. 



79. The apparatus of claim 65, wherein: 
additional fields are provided in respective subsequent 
VOL headers to designate respective subsequent buffer 
occupancy levels. 
5 80. The apparatus of claim 65, wherein: 

said pushed data flow comprises video data that includes 
intraframe (I), prediction (P) and bi-directional (B) 
video object planes ( VOPs ). 
81. A method for providing a push dataflow bitstream at 
10 a decoder, comprising the steps of: 

providing a data buffer of predetermined memory size at 
the decoder: and 

receiving the push dataflow bitstream at the decoder, 
t5 wherein said bitstream is obtained by encoding data in 
accordance with a model of the buffer so that the 
modeled buffer does not overflow or underflow; 
wherein: 

the encoded data comprises at least one video or visual 
20 object (VO) with at least one video or visual object 

layer (VOL), including an associated header, fol- 
lowed by at least one video or visual object plane 
(VOP); and 

a field in the VOL header designates an occupancy of 
25 the modeled buffer just before removal of the first 

VOP following the VOL header from the modeled 
buffer. 



82. A bitstream for push dataflow communication to a decoder 
having a data buffer of predetermined memory size, said 
bitstream being obtained by encoding data in accordance with 
a model of the decoder buffer so that the modeled buffer 
does not overflow or underflow; wherein: 

the encoded data of the bitstream comprises at least 
one video or visual object (VP) with at least one video or 
visual object layer (VOL), including an associated header, 
followed by at least one video or visual object plane (VOP); 
and 

a field in the VOL header designates an occupancy of 
the modeled buffer just before removal of the first VOP 
following the VOL header from the modeled buffer. 

83. The bitstream of claim 82, wherein: 

the encoded data further comprises a flag that controls 
the inclusion of at least one field in the VOL header when 
equivalent information is not present in an encapsulating 
system multiplex. 

84. The bitstream of claim 83, wherein: 

the flag allows a visual elementary stream of the 
bitstream as a standalone entity to specify a buffer model. 

85. The bitstream of claim 83, wherein: 

the at least one field whose inclusion in the VOL 
header is controlled by the flag designates a VOP rate of 
the bitstream. 

86. The bitstream of claim 83, wherein: 

the at least one field whose inclusion in the VOL 
header is controlled by the flag designates a peak bit rate 
of the bitstream. 



87. The bitstream of claim 83, wherein: 

the at least one field whose inclusion in the VOL 



header is controlled by the flag designates whether the VOL 
contains at least one B-VOP. 

88. The bitstream of claim 83, wherein: 

the at least one field whose inclusion in the VOL 
header is controlled by the flag designates the size of the 
modeled buffer. 

89. The bitstream of claim 83, wherein: 

the at least one field whose inclusion in the VOL 
header is controlled by the flag designates said field that 
designates the occupancy of the buffer. 

90. The bitstream of claim 82, wherein: 

the video or visual object comprises one of a video 
object, a still texture object, a mesh object, or a face 
object . 

91. The bitstream of claim 82, wherein: 

the encoded data comprises a plurality of VOLs, and a 
decoder buffer model is applied independently to each VOL 
using buffer size and rate functions particular to each VOL. 

92. The bitstream of claim 82, wherein: 

the bitstream is compatible with an MPEG-4 video coding 
standard. 

93. The bitstream of claim 82, wherein: 

addiLiunai fieius are provided In respective subsequent 
VOL headers to designate respective subsequent buffer 
occupancy levels. 

94. The bitstream of claim 82, wherein: 

said encoded data comprises video data that includes 
intraframe (I), prediction (P) and bi-directional (B) video 
object planes (VOPs) . 



